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We analyze 230.4 fb ^ of data collected with the BABAR detector at the PEP-II e+e col- 
lider at SLAC to search for evidence of D^-D" mixing using regions of phase space in the de- 
cay D° K+ n . We measure the time-integrated mixing rate Rm = (0.023 oi4 (stat.) ± 
0.004 (syst.))%, and Rm < 0.054% at the 95% confidence level, assuming CP invariance. The data 
are consistent with no mixing at the 4.5% confidence level. We also measure the branching ratio for 
D° K+TT-Tv° relative to D° K-tv+tt° to be (0.214 ± 0.008 (stat.) ± 0.008 (syst.))%. 

PACS numbers: 14.40.Lb, 13. 25. Ft, 12.15.Mm, ll.30.Er 



Mixing of the strong eigenstates and in- 

volving transitions of the charm quark to a down-type 
quark, is expected to have a very small rate in the 
Standard Model (SM). Accurate estimates of this rate 
must consider long-distance effects [H, and typical the- 
oretical values of the time-integrated mixing rate are 
Rm ~ O(10~^-10~^). The most stringent constraint to 
date is Rm < 0.040% at the 95% confidence level 0. 
Because SM D mixing involves only the first two quark 
generations to a very good approximation, the mixing- 
amplitude scale is set by flavor-SU(3) breaking, and CP 
violation is undetectable [2] . 

We search for the process jZ?*^) by analyz- 

ing the decay of a particle known to be created as a 
\D°) 0. We reconstruct the wrong-sign (WS) decay 
D'^ ii'+7r~7r°, and we distinguish doubly Cabibbo- 
suppressed (DCS) contributions from Cabibbo-favored 
(CF) mixed contributions in the decay-time distribution. 
Because mixing amplitudes are small, the greatest sensi- 
tivity to mixing is found when the amplitude for a par- 
ticular DCS decay is comparably small. We increase our 
overall sensitivity to mixing by selecting regions of phase 
space (i.e., the Dalitz plot) where the relative number of 
DCS decays to CF decays is small. This technique can- 
not be performed with the two-body decay D° — > if +7r~, 
and it has not been used to date. While the ratio of 
DCS to CF decay rates depends on position in the Dalitz 
plot, the mixing rate does not. From inspection of the 
Dalitz plots, we note that DCS decays proceed primarily 
through the resonance D'^ K*^tt^ , while CF decays 
proceed primarily through Z?" [A\. 

We present the first search for D mixing in the de- 
cay D'^ K^-k~'k'^ . The analysis method introduced in- 
creases experimental accessibility to interference between 
DCS decay and mixing without a full phase-space pa- 
rameterization. Such interference effects can be used to 
search for new physics contributions to CP violation. 

The two mass eigenstates 

\DA,B)^p\D'^)±qm (1) 

generated by mixing dynamics have different masses 
(m^.s) and widths {T a,b), and we parameterize the mix- 
ing process with the quantities 

If CP is not violated, then \p/q\ — 1. For a nonlep- 
tonic multibody WS decay, the time-dependent decay 



rate, r-ws(i)j relative to a corresponding right-sign (RS) 
rate, rRs(i)j is approximated by [5] 

p§^Rn+ay'^nin)+'-:^irtf (3) 

iRS(Ej 4 

< a < 1. 

The tilde indicates quantities that have been integrated 
over any choice of phase-space regions. Rd is the inte- 
grated DCS branching ratio, y' — ycos6 — xsinS and 
x' = xcosS + ysind, where S is an integrated strong- 
phase difference between the CF and the DCS decay 
amplitudes, a is a suppression factor that accounts for 
strong-phase variation over the regions, and F is the av- 
erage width. The time-integrated mixing rate Rm — 
(x'^ -|-y'^)/2 — {x'^ +y^)/2 is independent of decay mode. 

We search for CP-violating effects by fitting to the 
Z?" X+7r~7r" and K~tt~^tt'^ samples separately. 

We consider CP violation in the interference between the 
DCS channel and mixing, parameterized by an integrated 
CP-violating-phase difference (p, as well as CP violation 
in mixing, parameterized by \p/q\- We assume CP invari- 
ance in the DCS and CF decay rates. The substitutions 

cty' \p I q\^^ {ay' cos (j) ± (3 x' sincj)) (4) 
{x^+y^)^\p/qf\x^+y^) (5) 

are applied to Eq. [31 using (-I-) for 
T(D^ K~Ti+TT^)/T{D° K-TT+TT°) and (-) for 
the charge-conjugate ratio. The parameter /3 is a 
suppression factor that accounts for variation in the 
selected regions. 

We use 230.4 fb~^ of data collected with the BABm de- 
tector at the PEP-II e+e" colhder at SLAC. The pro- 
duction vertices of charged particles are measured with a 
silicon-strip detector (SVT), and their momenta are mea- 
sured by the SVT and a drift chamber (DCH) in a 1.5 T 
magnetic field. Particle types are identified using energy 
deposition measurements from the SVT and DCH along 
with information from a Cherenkov-radiation detector. 
The energies of photons are measured by an electromag- 
netic calorimeter. All selection criteria were finalized be- 
fore searching for evidence of mixing in the data. Selec- 
tion criteria were determined from both study of the RS 
sample and past experience with other charm samples 0] ■ 

We reconstruct the decay D*+ D^i^t and deter- 
mine the flavor of the candidate from the charge of 
the low-momentum pion denoted by irf- . We require tt^ 



5 



Wtd. Cand. 

10^ 





0.5 bd^'!^ 



wtd. Cand. 

10 



10-^ 



1 2 3 

•m2(A'-7r°) [GeV/c^]- 



FIG. 1: Dalitz plots and projections for RS (left) and WS 
(right) data. An additional selection is made to reduce peak- 
ing background in the events shown here, and no at selec- 
tion is made. A statistical background subtraction flfj and a 
phase-space dependent efficiency correction have been applied 
(i.e., candidates have been weighted). 

candidates to have momentum transverse to the beam 
axis pt > 120MeV/c. We require candidates to have 
center-of-mass momenta greater than 2.4GeV/c, and the 
charged daughters must satisfy a likelihood-based 
particle-identification selection. The identification effi- 
ciency for both K and tt is 90%, and the misidentifi- 
cation rate is 3% (1%) for K (tt) candidates. We re- 
quire photons from 7r° decays to have a laboratory energy 
Ej > 100 MeV, and tt'^ candidates to have a laboratory 
momentum p^o > 350 MeV/ c and a mass-constrained-fit 
X' 



probability > 0.01. The experimental width of the tt"- 



mass peak is ct„ 



■(77) 



6MeV/c . We accept candidates 



with an invariant mass 1.74 < mj^-^j^o < 1.98GeV/c^ and 
an invariant mass difference 0.140 < Am < 0.155 GeV/c^, 



where Am 



We enhance contribu- 



tions from D K~ and reduce the ratio of DCS to 
CF decays by excluding events with two-body invariant 
masses in the ranges 850 < m{K'K'^ , Ktt^) < 950MeV/c^. 
Figure [1] shows the Dalitz plots for these decays. 

The D*+ mass, mass, and Z?" decay time are de- 
rived from a track- vertex fit Q. A mass constraint is 
applied to the 7r° candidate, and the D*+-decay vertex is 
constrained to the beamspot region, of size (cFx, fj,, cr^) w 
(150 /im, 10 /im, 7 mm). We select events for which the fit 
probability > 0.01. From this fit, a D° decay time, 
t, and uncertainty, at, are calculated using the three- 
dimensional flight path. The full covariance matrix, in- 
cluding correlations between the D*~^ and D'~' vertices, 
is used in the at estimate. For signal events, the typi- 
cal value of at is near 0.23 ps. We accept decays with 
at < 0.5 ps. The D° lifetime is (410.1 ± 1.5) fs [g. 




1.8 1.9 

mK+n-n« [GeV/c2] 



0.14 0.15 

Am [GeV/c2] 




-10 12 3 
t[ps] 



10 12 3 

MPS] 



FIG. 2; Distributions of WS data (points with error bars) 
with fitted PDFs (dashed line) overlaid. The rriK^^o dis- 
tribution (a) requires 0.1444 < Am < 0.1464 GeV/c^; the 
Am distribution (b) requires 1.85 < mj^^^o < 1.88GeV/c^; 
and the t distribution (c) requires both mass selections. The 
data points in (d) show the t distribution after applying a 
channel- likelihood signal projection [HI . [l^ . and the signal 
PDF is overlaid. The error bars in (d) refiect Poissonian 
signal fluctuations only. In (a)-(d), the white regions repre- 
sent signal events, the light gray misassociated tt^ events, the 
medium gray correctly associated tt^ with misreconstructed 
events, and the dark gray remaining combinatorial back- 
ground. 



We first extract the signal yields from a two- 
dimensional, unbinned, extended maximum likelihood fit 
to the mj^^^o and Am distributions, performed on the 
RS and WS samples simultaneously. The signal-shape 
parameters of the probability density function (PDF) de- 
scribing the WS sample are precisely determined by the 
large RS sample, and all associated systematic uncertain- 
ties are suppressed. The width of the Am peak is uncor- 
related with the width of the mx-mr" peak, dominated by 
TT^-momentum resolution, to first order. However, there 
is a second-order correlation in the signal between the 
two distributions. Thus, the signal PDF has a width in 
Am that varies quadratically with mj^^^o . This feature 
signiflcantly reduces the signal yield uncertainty. 

Three background categories are included in the like- 
lihood: (1) correctly reconstructed candidates with a 
misassociated 7r+, (2) D*+ decays with a correctly asso- 
ciated 7r+ and a misreconstructed and (3) remaining 
combinatorial backgrounds. The first category has distri- 
butions in mKTnrO and t of RS signal decays and is distin- 
guished using Am. The second category, peaking in Am 
and distinguished using mx-n-K° j has a t distribution sim- 
ilar to RS signal with a different characteristic lifetime. 
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TABLE I: Signal-candidate yields determined by the two- 
dimensional fit to the (m^^^o , Am) distributions for the WS 
and RS samples. Yields are shown (a) for the selected phase- 
space regions used in this analysis and (b) for the entire al- 
lowed phase-space region. Uncertainties are those calculated 
from the fit, and no efficiency corrections have been applied. 



Cand. 



Cand. 



WS 



(3.84 ± 0.36) X 10^ 



(3.79 ± 0.36) X 10^ 



RS (2.518 ± 0.006) X 10^ (2.512 ± 0.006) x 10^ 



WS 



(7.5 ±0.5) X 10^ 



.1 ±0.5) X 10^ 



RS (3.648 ± 0.007) x lO'^ (3.646 ± 0.007) x 10^ 



The third category does not peak in either m^^^o or Am 
and has a t distribution empirically described by a Gaus- 
sian with a power-law tail. Although the functional forms 
of the background PDFs are motivated by simulations, 
all shape parameters are obtained from a fit to the data. 
The mjf^^o and Am projections of the two-dimensional 
fit to the WS sample are shown in Fig. [2l[a,b) . 

The signal yields from the fit to the (m/f^jro i Am) 
plane are listed in Table [J Considering the entire al- 
lowed phase space, and without the at selection, we mea- 
sure the branching ratio for D'^ — > i^+7r~7r° relative to 
the decay D° -> K-tt+tt" to be (0.214 ± 0.008 (stat.) ± 
0.008 (syst.))%. This result is consistent with previous 
measurements (lOj of this quantity and is significantly 
more precise. For this measurement, a phase-space de- 
pendent efficiency correction is applied to account for the 
different resonant populations in CF and DCS decays. 
The average efficiency of the WS sample relative to the 
RS samples is 97%. Phase-space dependent 7r° selection 
efficiencies dominate the systematic uncertainty. 

The fitted shape parameters from mx-KTT° ^^nd Am are 
used to determine the signal probability of each event in 
a three-dimensional likelihood, £, that is optimized in 
a one-dimensional fit to t. The RS signal PDF in t is 
represented by an exponential function convolved with a 
three-Caussian detector-resolution function. The Gaus- 
sians have a common mean, but different widths. The 
width of each Gaussian is a scale factor multiplied by fit, 
and at is determined for each event. The three different 
scale factors, as well as the fraction of events described by 
each Gaussian, are determined from the fit to the data. 
We find a D''^ lifetime consistent with the nominal value. 

The WS PDF in t is based on Eq. [3] convolved with 
the same resolution function as in the RS PDF. The 
lifetime and resolution scale factors, determined by the fit 
to the RS t distribution, are fixed. We fit the WS PDF to 
the t distribution allowing yields and background-shape 
parameters to vary. The fit to the t distribution is shown 
for the WS sample in Fig. [2llc,d). 

The results of the decay-time fit, with and without 
the assumption of CP conservation, are listed in Ta- 
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FIG. 3: Contours of constant A\n£, = 1.15, 3, defining 68.3% 
and 95.0% confidence levels, respectively. The contours on the 
left are in terms of the integrated mixing rate, Rm, and dou- 
bly Cabibbo-suppressed rate, Rd, assuming CP invariance. 
The contours on the right are in terms of Rm andthe nor- 
malized interference I = {ay' cos 4>±j3x' sin (j))/ yjx^ + y^, for 
the and D" samples separately. On the left, the upward 
slope of the contour indicates negative interference; on the 
right, the hatched regions are physically forbidden. 

ble ini The statistical uncertainty of a particular param- 
eter is obtained by finding its extrema for AlnC — 0.5. 
Contours of constant Aln£ 1.15,3, enclosing two- 
dimensional coverage probabilities of 68.3% and 95.0%, 
respectively, are shown in Fig. [31 With a Bayesian inter- 
pretation of £, we find an upper limit < 0.054% at 
the 95% confidence level, assuming CP conservation. 

In one dimension, Aln£ changes its behavior near 
Rm = because the interference term (the term lin- 
ear in t in Eq. [3]) becomes unconstrained. Therefore, we 
estimate the consistency of the data with no mixing us- 
ing a frequentist method. We generate 1000 simulated 
data sets with no mixing but otherwise according to the 
fitted PDF, each with 58,800 events representing signal 
and background in the quantities mxTT-K° , Am, and t. 
We find 4.5% of simulated data sets have a fitted value 
of Rm greater than that observed in the data. Thus, the 
observed data are consistent with no mixing at the 4.5% 
confidence level. 

We quantify systematic uncertainties by repeating the 
fits with the following elements changed, in order of sig- 
nificance: the background PDF shape in the ttikt^t^o dis- 
tribution, the selection of events based on at , the decay- 
time resolution function, and the measured D*^ lifetime 
value. Additionally, for Rd, we consider the absence of 
any Dalitz-plot efficiency correction. The combined sys- 
tematic uncertainties are smaller than statistical uncer- 
tainties by factors of 2-4. The quantity /3a;' sin (j>, which 
quantifies a difference between the and D'^ samples, 
has a negligible systematic uncertainty because positively 
correlated effects in the two samples cancel. 

As a consistency check, we perform the decay-time fit 
to the entire phase-space region populated by the decays 
D'^ K^TT^-K^ . The results are consistent with Table llll 
with sensitivity to Rm preserved. However, the interfer- 
ence term obtained is different. Figure [3] indicates that 
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TABLE II: Mixing results assuming CP conservation and 

samples arc not separated) and manifestly permitting CP 
violation (D" and D" samples are fit separately). The first 
listed uncertainty is statistical, and the second is systematic. 
Quantities that have been integrated over the selected phase- 
space regions axe indicated with tildes. Rd is not reported 
when allowing for CP violation because precise T^f efficiency 
asymmetries are unknown. 



CP conserved 


CP violation allowed 


Rm (0.023 tolfi ± 0.004)% 
Rd (0.164 t°;°26± 0.012)% 


(0.010 ± 0.003)% 


ay' -0.012 t^ml ± 0.002 


ay' cos 4> 

-0.012 toZj ± 0.002 
fix sin (p 

0.003 Ig-.^gg ± 0.000 




\p/q\ 2.2+_\l±Q.l 



both D° and D'^ samples prefer a large negative interfer- 
ence term when the phase space is restricted to suppress 
DCS contributions. By contrast, when the interference 
term is integrated over the entire Dalitz plot, it is found 
to be consistent with zero, with uncertainties comparable 
to those in this analysis. The variation of the interfer- 
ence effect in different phase-space regions motivates a 
detailed phase-space analysis of this mode in the future. 

In summary, we find that the data are consistent with 
the no-mixing hypothesis at the 4.5% confidence level, 
and we set an upper limit Rm < 0.054% at the 95% 
confidence level. We measure the branching ratio for 
£)0 ^ K+TT-TT^ relative to ^ R-n+TT^ to be (0.214 
± 0.008 (stat.) ± 0.008 (syst.))%. 
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